Analysis of Correlated Expert Judgments from Extended Pairwise Comparisons

We develop a Bayesian multivariate analysis of expert judgment elicited using an extended form of pair-wise comparisons. The method can be used to estimate the effect of multiple factors on the probability of an event and can be applied in risk analysis and other decision problems. The analysis provides variance predictions of the quantity of interest that incorporate dependencies amongst the various experts. Unlike other combination methods for expert judgment, in this form we may learn about the dependencies between the experts from their responses. The analysis is applied to a data set of expert judgments elicited during the Washington State Ferries Risk Assessment. The effect of the statistical dependence amongst experts is compared to an analysis assuming independence amongst them.